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Abstract 

The understanding of complex, dynamic and animate systems has a special standing in education for sustainable 
development and biology. Thus one important role of science teacher education is to promote student teachers’ 
Content Related Knowledge (CRK) for teaching systems thinking, consisting of extensive Content Knowledge 
(CK) and well formed Pedagogical Content Knowledge (PCK). But the effective development of this knowledge 
is empirically uncertain. This article reports the effects of three different interventions (technical course, didactic 
course, mixed course) in promoting student teachers’ reflective Content Related Knowledge for teaching systems 
thinking. Participants analyzed a videotaped lesson of 6th graders learning about the ecosystem lake and 
identified weak aspects in the teacher’s CK and PCK. The results revealed that student teachers’ reflective 
Content Related Knowledge for teaching systems thinking can be promoted in teacher education. The conclusion 
to be drawn from our findings is that a mixed course with equal technical and didactical input seems to be most 
effective in fostering student teachers’ reflective Content Related Knowledge for teaching systems thinking. 

Keywords: Pedagogical Content Knowledge, Content Knowledge, systems thinking, sustainable development, 
teacher education 

1. Introduction 

The teacher is the most important factor in student learning (Abell, 2007; Hattie, 2009). During the last 30 years, 
research in teacher education has searched for the essential characteristics possessed by a good teacher. To 
pursue this inquiry, researchers have used three different paradigms: the teacher personality paradigm , the 
process-product paradigm , and more currently the expert paradigm (Fischer et al., 2012). In early studies, 
Shulman and colleagues (Shulman, 1987) investigated teachers’ professional knowledge. Research after 
Shulman (1986, 1987) began to draw a distinction between Pedagogical Knowledge (PK) and Content Related 
Knowledge (CRK). Content-Related Knowledge contains domain-specific subject-matter knowledge, also called 
Content Knowledge (CK), and the knowledge needed for teaching a specific content, called Pedagogical Content 
Knowledge (PCK). Pedagogical Knowledge (PK) refers to the non-content related educational knowledge of 
strategies and procedures to allow effective and undisturbed lessons. This latter type of knowledge goes beyond 
the subject matter and was not a part of our study. 

In our larger study SysThema (systems thinking in ecological and multidimensional areas), we investigated in 
four substudies effects of different interventions on aspects of student teachers’ teaching competencies. Figure 1 
provides a summary of the different substudies in the SysThema study. This article focusses on student teachers 
CRK for teaching systems thinking. To evaluate student teachers professional knowledge in a situational context, 
we created a video-based measurement instrument. In comparison with other measurement instruments such as 
multiple-choice tests, open-ended pen and paper tests or teacher’s observations, the evaluations of videotaped 
lessons offer the possibility to uncover the complexity of teacher’s professional knowledge in action (Seidel & 
Prenzel, 2007; Sherin & van Es, 2009). 
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Figure 1. Substudies in the larger SysThema study 


1.1 Teacher's Competencies 

The distinction of the different forms of teachers’ CRK has been widely embraced in research on teacher 
knowledge (Abell, 2007; Borowski et al., 2012; Krauss et al., 2008; Kunter & Baumert, 2013). The conclusion is 
that CK and PCK may be hypothesized to represent conceptually distinct forms of knowledge, with CK being 
the requisite for the development of PCK (e.g., Ball et al., 2005; Kunter & Baumert, 2013). Given this current 
accepted model of teachers CRK, we characterise the different forms separately. 

Many different conceptualisations of teacher’s essential CK exist in the literature (e.g., Ball et al., 2005; Kunter 
& Baumert, 2013). Nevertheless, common to all models is a deep understanding of CK, which goes beyond the 
common knowledge possessed by well-educated adult. This knowledge base implies not only knowing that 
something is so, but also the understanding why it is so (Shulman, 1986). How well teachers know their content 
“is central to their capacity to use instructional materials wisely, to assess students’ progress, and to make sound 
judgments about presentations, emphasis, and sequencing” (Ball et al., 2005, p. 14). Thus it is not surprising that 
several studies provide evidence for a positive relationship between the teacher’s CK and the respective students’ 
learning progress (Abell, 2007; Baumert et al., 2009; Hashweh, 2005). 

PCK “goes beyond knowledge of subject matter perse to the dimension of subject matter knowledge for 
teaching” (Shulman, 1986, p. 9). Hence, PCK is the “result of a transformation of knowledge from other 
domains” (Magnusson et al., 1999, p. 96), which implies both a CK base and a PK base. Further studies have 
emphasised the essential need for CK for the development of PCK (Abell, 2007; Borko & Putnam, 1995; 
Grossman, 1990; GroBschedl et al., 2014; Kunter & Baumert, 2013; Magnusson et al., 1999; Park & Oliver, 
2008; Van Driel et al., 1998). Teachers possessing extensive PCK know the most powerful forms of 
representations and the most powerful analogies, illustrations, examples, explanations, and demonstrations. PCK 
also includes the knowledge and understanding of factors that make the learning of a special topic easy or 
difficult (Shulman, 1986). These components are essential parts of PCK and are included in almost every 
conceptualisation. Knowledge of curriculum and knowledge of educational ends are two additional forms 
considered in many conceptualisations of PCK. An overview of individual conceptualisations is given by Park 
and Oliver (2008), and Van Driel, Verloop, and De Vos (1998). Overall, research in teacher education has 
explored the described forms of CRK as main factors for effective teaching and learning (Krauss et al., 2008; 
Baumert et al., 2009). 
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1.2 Knowledge for Teaching Systems Thinking 

Future teachers need a well-formed CRK to be able to teach about the complexity of a world getting increasingly 
global and interconnected and to foster an understanding of complex and dynamic systems. A system is defined 
as a “set of elements standing in interrelation” (Bertalanffy, 1968, p. 55). The system theory approach explains 
the behavior of complex systems, e.g., of ecosystems from a certain point of view. The aim to promote systems 
thinking in school is based inter alia on the assumption that students can only actively participate in sustainable 
development when they are able to identify and to understand complex, global relations using the methods of 
system theory (Riess & Mischo, 2010). Our consideration of systems thinking is based on the definition of Riess 
and Mischo (2010, p. 707): 

We see systems thinking as the ability to recognize, describe and model (e.g., to structure, to organize) 
complex aspects of reality as systems. Another important aspect of systems thinking is the ability to 
identify important elements of the system and the varied interdependency between these elements. Other 
key aspects are the ability to recognize dimensions of time dynamics, to construct an internal model of 
reality and to make prognoses on the basis of that model. 

To conceptualise this definition in more detail, RieB and his research group developed a heuristic competence 
model in systems thinking and used it in the survey of Brautigam (2014). Brautigam had investigated the 
systems thinking skills of 7th graders and used a three dimensional competence model. Errors and insufficiencies 
of the competence model noted in her study were discussed and revised. On the basis of further studies in the 
literature in systems sciences (e.g., Matthies et al., 2001), the analysis of other competence models in systems 
thinking (Ben-Zvi Assaraf & Orion, 2005; Evagorou et al., 2009; Rempfler & Uphues, 2011; Verhoeff, 2003), 
including several discussions with researchers in environmental systems sciences, we added an further aspect to 
illustrate the relevance of evaluating the model and validating the results. Thus the current heuristic competence 
model of systems thinking (RieB, Schuler, & Horsch, 2015) includes four dimensions, each divided into four 
sub-capabilities (see Figure 2). 


Competence Dimensions 

Sub-Capability 1 

Sub-Capability 2 

Sub-Capability 3 

Sub-Capability 4 

Dimension 4: 

evaluation of system models 

determining the structural validity 
of system models 

determining the validity of system 
models 

determining the validity of the 
application 

determining the uncertainty of a 
forecast 
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solving problems 
using system models 

assessing the need of using a 
system model for processing a 
present problem 

assessing the type of system 
model (e.g. quantitative vs. 
qualitative) that is required to 
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giving explanations, making 
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technologies based on qualitative 
system models 

giving explanations, making 
predictions and designing 
technologies based on quantitative 
system models 

Dimension 2: 

modeling systems 

determining system elements, 
interactions, subsystems, system 
boundaries, system hierarchies 
and the model purpose 

understanding and reflecting a 
complex system with the help of a 
text field or a word model 

reading and understanding 
qualitative system models. 

Construct influence diagram 

reading and constructing 
quantitative system models 

Dimension 1: 

declarative / conceptual systems 
knowledge 

basic knowledge of systems theory 
(system concept, system structure, 
system behavior, sub-systems) 

knowledge of areas that can be 
considered as systems (also 
knowledge of simple and complex 
systems) 

knowledge of system hierarchies 
(e.g., cell, tissue, organ, organism, 
population, biocenosis, ecosystem, 
biosphere) 

knowledge of properties of 
complex systems (structural and 
dynamic complexity, non-linearity, 
emergence,....) 


Figure 2. Freiburg heuristic competence model of systems thinking 


The conceptualisation of student teachers’ CRK for teaching systems thinking is composed of CK for teaching 
systems thinking and PCK for teaching systems thinking, which are elaborated in the following section. 

1.3 Conceptualisation of Teacher’s CK about Ecology and Systems Sciences 

When conceptualising teachers’ CK for teaching systems thinking, we orientated on the described heuristic 
competence model of systems thinking (see Figure 2). However when theorising about CK, we need to 
distinguish static knowledge about ecological facts and concepts and principles that apply within systems 
sciences (declarative knowledge) from procedural knowledge, which contains actions or manipulations that are 
valid within the systems sciences (De Jong & Ferguson-Hessler, 1996). Declarative knowledge about ecology 
and systems sciences is described in Dimension 1 (declarative/conceptual systems knowledge) and Dimension 2 
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(modeling systems). Dimension 1 includes basic knowledge of systems theory, knowledge of areas that can be 
considered as systems, knowledge of systems hierarchies and knowledge of properties of complex systems. 
Competences in modeling systems are required in Dimension 2, such as determining system elements, 
interactions and subsystems but also the understanding of and the ability to reflect on complex systems with the 
help of a text field or a word model, as well as reading and understanding both qualitative and quantitative 
system models. Procedural knowledge in solving problems by using system models is listed in the third 
dimension. Teachers need to know how to assess whether a system model is useful for processing a problem and 
whether a quantitative or a qualitative model is required, as well as knowing how to provide explanations, make 
predictions, and design technologies based on these models. Also required in teachers’ CK is the ability to 
evaluate system models according to their validity (e.g., in structure and in application) and the uncertainty of a 
prediction. 

1.4 Conceptualisation ofPCK for Teaching Systems Thinking 

CRK includes the special knowledge for teaching, PCK, which goes beyond subject matter knowledge. Based on 
the approaches described, we conceptualized PCK for teaching systems thinking in three facets, as follows: 

Knowledge of Curriculum and Educational Ends includes the “teacher’s knowledge of the goals and objectives 
for students” (Magnusson et al., 1999, p. 103) in promoting systems thinking. The teacher needs to know about 
the heuristic competence model in teaching systems thinking (see Figure 2) and they also need to know the 
competencies that are essential for teaching. Peterson and Treagust (1995) found that knowledge of curriculum 
was an essential part of pre-service teachers’ pedagogical reasoning about lesson planning and instruction. The 
teacher, possessing a substantial PCK, knows the curriculum and educational ends for teaching. 

Knowledge of Instructional Strategies means knowing about topic specific methods, representations, examples, 
models and activities (Abell, 2007; Shulman, 1986). It includes knowledge of methods as well as their 
reasonable and productive use (Magnusson et al., 1999) with the goal of fostering systems thinking. Teachers 
know the “most powerful strategies” (Shulman, 1986, s. 9) and their use for fostering systems thinking. They 
have a repertoire of methods for promoting declarative and conceptual systems knowledge such as working with 
worksheets about systems sciences or using of educational films about ecosystems and their integration in 
lessons. They know the best teaching methods and strategies to enable pupils to model systems, solve problems 
and evaluate models. 

Knowledge of Students ’ Understandings characterises the ability of teachers to understand the perspective of 
their pupils’ position. From this perspective, teachers can discern pupils’ correct and incorrect conceptions 
concerning systems thinking as well to assess “what makes the learning of specific topics easy or difficult” 
(Shulman, 1986, p. 9). Substantial and correct understanding in systems thinking is needed to be able to not only 
recognize that a pupil’s answer might be wrong but also to analyze the source of the error. 

The relevance of teachers’ CRK has been discussed in several studies, particularly in mathematics and science 
teaching (Ball, Lubienski, & Mewborn, 2001; Krauss et al., 2008; GroBschedl et al., 2014). However, the 
effective development in teacher education with the focus on the key comptence in sustainable development of 
CRK for systems thinking, remains empirically unclear. 

2.Method 

2.1 Context of the Study and Research Question 

The goal of the present study was to evaluate the effects of three different interventions in teaching systems 
thinking for student teachers at the Universities of Education in Freiburg and in Ludwigsburg (Germany) during 
the summer semester 2013. We used a quasi-experimental approach (Shadish et al., 2002) with three treatment 
courses and a control group to examine effects of different inputs on the dependent variable, student teachers’ 
CRK. Every course consisted of 14 sessions of 90 minutes each. All of the courses had two common goals. First, 
the courses were designed to enhance student teachers’ ability in systems thinking, that is, their ability to solve 
complex dynamic problems within the context of sustainable development (acquiring CK). Second, the 
participants should gain the ability to teach systems thinking effectively (acquiring PCK). In Course 1, student 
teachers received a more technically oriented input, whereas they received a more subject-related didactic input 
in Course 2. Course 3 combined technical and subject-related didactic content in almost equal proportions. The 
Control Group received no intervention. We used these variations of the intervention to clarify which 
combination of technical and subject related didactic content helps student teachers most effectively in teaching 
systems thinking. 
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Figure 3. Overview of the study (we used the video in all measurements to capture student teachers’ reflective 

CRK) 


This design opens the possibility to “test descriptive causal hypotheses about manipulable causes” (Shadish et al., 
2002, p. 14). For each group, CRK for teaching systems thinking (dependent variable) was measured before the 
intervention (pretest, tl), and after the intervention (posttest, t2). Two weeks after the intervention, we measured 
again in a follow-up test (follow up test, t3). 

As an alternative to traditional pen & paper tests, videos showing an authentic situation in class are commonly 
used. The participants’ task was to analyze the presented video and to note key aspects indicating good teaching, 
such as respecting pupils’ cognitions or using instructional strategies adequately. The ability to judge the 
teaching in the video required that the teacher had a professional vision (Todorova, Sunder, Steffensky, & Moller, 
2014), that is, the ability to identify and interpret important aspects in educational situations. Professional vision 
functions as a mediator between theory and the teacher’s practice (Schmelzing et al., 2013) and is an essential 
requirement for the teacher’s adaptive action (Bromme, 2008; Sherin & van Es, 2009). Schmelzing et al. use the 
term reflective professional knowledge. This knowledge is necessary for identifying habits based on pedagogical 
actions and for analyzing the effects of teacher actions (Schmelzing et al., 2013). 

Based on this theoretical framework, we postulated the following questions: 

(1) Is there an intervention effect in student teachers’ reflective CRK in evaluating a lesson with the goal of 
fostering systems thinking? 

(2) Are there identifiable differences in the intervention courses? Which seminar is most effective in promoting 
student teachers’ reflective CRK in evaluating a lesson with the goal of fostering systems thinking? 

2.2 Participants 

We offered courses in geography and biology at universities of education in Germany. These courses were 
essential seminars in student teachers’ curriculum and were labeled without describing any specific information 
about the content. The participants (108 student teachers) selected the intervention courses blindly. All the 
participants had gained the German Abitur (A level degree qualifying for university admission), studied on 
average in their 6th semester, and were aged between 20 and 47 years (see table for descriptive statistics). The 
low representation of males corresponds with the usual relations of female and male student teachers in German 
universities of education. 


Table 1. Descriptive statistics of intervention and control groups 



Control Group 

(1) Technical Course 

(2) Didactic Course 

(3) Mixed Course 

N 

37 

23 

25 

23 

Male/Female 

4/33 

5/18 

0/25 

6/17 

0 grade 

2.38 (5X1=0.54) 

2.57 (5X>=0.45) 

2.46 (5X>=0.39) 

2.50 (5X>=0.42) 

0 age 

23.68 (5X>=2.41) 

23.41 (5D= 1.7 9) 

24.64 (5X>=5.03) 

24.52 (5X>=4.57) 

0 semester 

6.19(SX>=2.03) 

5.78(5X1=1.95) 

6.68 (5X>=1.67) 

6.09(5X1=1.38) 


160 

























hes.ccsenet.org 


Higher Education Studies 


Vol. 6, No. 4; 2016 


2.3 Instruments 

To measure student teachers’ reflective CRK, we developed a video with an associated booklet. While content of 
the intervention courses was about the ecosystem forest and its complexity and dynamic in a sustainable 
development, the video to measure student teachers’ reflective CRK was about the ecosystem lake, which 
required the use of systems thinking in a transferred content. The target pupils were 6th graders, because they 
learned about complex and dynamic ecosystems for the first time in German secondary school curriculum. The 
student teachers were introduced to the associated booklet about the procedure and the goals of the lesson. We 
installed six mistakes in the video vignette, three in each form of CRK. 

The procedure begins with an introduction to the ecosystems of a lake. The teacher provides an incorrect 
definition of a system (first CK aspect), labels the connections and interactions incorrectly (second CK aspect) 
and demonstrates the complexity of a system mainly as a food web (third CK aspect). In further situations, the 
teacher also shows three critical weaknesses in PCK. The teacher assumes pupils know the term system in 
ecological sciences when the topic is introduced (first PCK aspect). After the pupils read the incorrect definition 
already written on the board, they are asked to identify some interactions within the ecosystem of lakes. The 
teacher notes all the mentioned interactions on the board regardless of whether they are right or wrong (second 
PCK aspect). The interaction between pollywogs and frogs is labeled become to and the interaction between 
water lilies and water plants is labeled with the term belong to. Thus, a confusing map with many labeled 
connections and interactions arise. The pupils are encouraged to copy the model to their notebooks. In the end, 
the planned goals of introducing systems thinking with the example of the ecosystem lake and gaining basic 
knowledge of the system sciences by constructing a simple system model are not achieved (third PCK aspect). 
After watching the video, student teachers were requested to analyze and assess the lesson from a professional 
point of view and to make notes about the erromeous situations. Figure 4 shows an overview of the mistakes. 

To achieve satisfactory content validity, we asked a number of experts (researchers in the didactics of biology 
and geography) to investigate the video in an online survey. After the experts examined the video, they were 
asked to mark the critical aspects in the online option list. The experts had the option to make suggestions to 
optimise the measurement. We analysed both, the marked options in compliance with the critical aspects and the 
experts’ suggestions (e.g., omitting multiple choices). On the basis of these analyses, we improved the 
measurement instrument by omitting the choices to avoid anticipations and coincidences. 


Errors related to Teachers' Content Knowledge (CK) 

Errors related to Teachers' Pedagogical Content Knowledge (PCK) 

Incorrect definition of a system 

The assumption that pupils know the term system in ecological 
sciences 

Incorrect labeling of the connections and interactions 

Confusing map with many labeled connections and interactions n 
the board 

Complexity of a system is demonstrated mainly as a foodweb 

The planned goals are not achieved 


Figure 4. Overview of the mistakes in the video vignette 


2.4 Intervention 

Within the larger study, SysThema , we designed three different courses for student teachers in the subject areas 
of biology and geography (described more in detail in RieB, Schuler, & Horsch, 2015). The didactic structure of 
all units in the intervention follows the Model Of Problem-Oriented Learning and Teaching (MOPOLT), created 
by RieB and Mischo (2016). At the beginning of each unit, the student teachers were introduced to the goals of 
the session and their importance. They began by working on a task, for example analyzing a complex problem 
within a genuine case example or creating an effective lesson about systems thinking. The student teachers 
became engaged in the given task, and while trying to solve the problem, they realized their need for further 
knowledge and competencies. The lecturer as an expert demonstrated the method developed by scientists to 
solve the problem and explained the thoughts behind each step. The student teachers were introduced into the 
expert culture by authentic activities and specific instruction. Over the duration of the unit, the student teachers 
worked alone or in small groups on additional tasks similar to the first task while progressively learning to solve 
them independently. The results and the applied strategies were presented and reflected in class. Finally, the 
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student teachers were given the opportunity to further practice and to transfer and apply the gained competencies 
in other areas. 

In the first units, specific knowledge about systems was imparted in the following three topics: 1) introduction 
into systems sciences and analysis (Bossel, 2007; Matthies et al., 2001); 2) forest ecosystem; and 3) syndromes 
of global change, for example deforestation, soil degradation, overfishing, climate change, and overfishing 
according to the syndromes concept (Schellnhuber et al., 1997). Later in the courses, the case examples, systems 
modeling, systems analysis, and lesson planning were related to these topics. After providing a common 
foundation, the three courses diverged with regard to their technical and subject-related didactic proportions. 




Units 1-2 

Units 3-14 | 

Technical Course (1) 

Didactic Course (2) 

Mixed Course (3) 

specific knowledge 
about systems 
(introduction into 
systems sciences and 
analysis; ecosystem 
forest; syndromes of 
global change) 

Technical Input (modeling and analyzing complex systems) 

Didactic Input (teaching systems thinking in school effectively) 

Technical Input Didactic Input 

Control Group 

no intervention 


Figure 5. Overview about the intervention 


2.4.1 Intervention 1—Technical Course 

In the Technical Course, the different systems were modeled and analyzed with increasing autonomy and on a 
higher level of systems science. For example, the student teachers learned to construct an influence diagram such 
as a qualitative system model, consisting of system elements (nodes) and system relationships (influences, 
arrows). Quantitative system models were also constructed and simulated with software. The student teachers 
learned to analyze complex systems (e.g., the issue with overfishing; Kropp et al., 2006) to be able to provide 
explanations and to make predictions with regard to the behavior of a system and to find appropriate 
measurements to influence the system behavior in a positive way (e.g., calculating fishing quotas for sustainable 
fishing in a fishery model; Bossel, 2007). Furthermore, the student teachers evaluated system models, 
determined their validity, and developed an awareness of their uncertainty. The students determined the 
sensitivity of the system model and created a range of possible future scenarios, and then they tested their 
implications by running the simulation under different conditions. How to foster pupils in systems thinking in 
school and how to create an appropriate learning environment was less of a focus in the Technical Course. 

2.4.2 Intervention 2—Didactic Course 

The more subject-related didactic course emphasized the effective teaching and learning of systems thinking in 
school. The student teachers approached the question of what is actually meant by systems thinking. They 
compared and reflected on different conceptualizations found mainly in the German literature (e.g., RieB & 
Mischo, 2010). The reasons for teaching systems thinking to pupils were discussed in depth. The participants 
formulated learning goals on different levels, compared them with the existing competence model (see Figure 2; 
cf. RieB, Schuler, & Horsch, 2015), and created different tasks for measuring the competence of systems 
thinking. The student teachers learned about several teaching methods (e.g., Schuler, 2012), tested them, and 
planned numerous lessons. Scientific methods were considered, implemented, and reflected on as possible 
teaching methods. They also learned about pupils’ preconceptions of systems (e.g., Jelemenska & Kattmann, 
2008), and they thought about how to promote learning by engaging the pupils. Systems modeling and analysis 
received less attention and was practiced mostly with qualitative system models only. The student teachers 
worked with fewer case examples and were more guided throughout the process. For example, they operated 
with simpler, ready-made simulation models on the computer. Most importantly, the participants adopted the 
perspective of a teacher and evaluated, for example, the impact analysis as a potential teaching method. 
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2.4.3 Intervention 3—Mixed Course 

The third course combined technical and subject-related didactic content in almost equal proportions. The Mixed 
Course was designed from central components out of both courses described above. The participants almost 
worked on the same level of systems science as the student teachers in the technical course, but they mostly used 
only one case example. They had less opportunity to practice systems thinking autonomously and to transfer 
their knowledge to other domains. Furthermore, they evaluated system models only slightly and reflected a little 
on the validity and limits of the models. However the Mixed Course included sessions on conceptualizations of 
systems thinking, on a competence model, teaching methods, and pupils’ preconceptions. The participants in this 
course had little time to work with the teaching methods or to practice, reflect, and to integrate them in planned 
lessons. 

3. Results 

We analyzed student teachers’ notes by using a coding scheme according to Mayring (2007). We derived criteria 
for correct notes from the critical aspects described. On the basis of this coding scheme, three trained student 
assistants and the first author categorized the notes. Every correct comment was scored 1 and false answers were 
scored 0. Thus, the scoring ranged from 0 to a maximum of score of 6. After a few codings and some adaption of 
the coding scheme, we reached an acceptable interrater reliability (Krippendorff s alpha over .80; Hayes & 
Krippendorff, 2007). With the sum score added by all correct notes, we used the statistical software IBM SPSS 
v.22 for Windows for further analysis. This analysis belongs to the statistical mixed-methods analysis and 
represents a hybrid analysis using qualitative and quantitative methods. 

The reliability marks the precision of the measurement instrument, but due to the limited number of items, the 
calculation of the internal consistency with Cronbach’s a (Cronbach, 1951) is not meaningful (Schmitt, 1996). 
Another way to measure the reliability is through repeated testing. An reliable instrument should highly correlate 
with the previous testing (Trochim, 2000). In the present study, the correlation between the second and the third 
measurement indicates a reliable measurement (r=.583**). Expected correlations between this instrument and 
the paper and pencil measurements of declarative CK in teaching systems thinking (r=.313**; Fanta et ah, 2016) 
and the declarative PCK in teaching systems thinking (r=.488**; Rosenkranzer Stahl, et ah, 2016) were 
observed (convergent validity) but indicate a different form of knowledge measured (Trochim, 2000). 

Descriptive statistics were generated and the data were verified to be normally distributed. To analyze whether 
the experiment’s four conditions differ a priori in the dependent variable (sum score of the reflective CRK in 
teaching systems thinking), we tested the assumption of equal variances by computing Levene’s test. An 
univariate analysis was performed with the reflective CRK at time tl and the independent variable “experimental 
condition” (Brown & Forsythe, 1974). The results of the test showed no significant difference between the four 
groups, F (3,104)=2.24, p=. 07. Nevertheless, given that the analysis showed some meaningful difference close to 
the significance level, we decided to adjust for differences by performing analyses of covariate (ANCOVA) in 
further tests. 

(1) Is there an intervention effect in student teachers’ reflective CRK in evaluating a lesson with the goal of 
fostering systems thinking? 

We assessed student teachers’ reflective CRK in evaluating a lesson with the goal of fostering systems thinking 
at three temporal measurement points. On average, the student teachers could identify only one of the six 
mistakes in the video before the intervention; M=1.25 (ND=1.08). After the intervention student teachers found 
more mistakes M=\.ll (SD=\A2) but without any statistical significances F (3,103)=2.34; p=. 08. In follow up 
test M=1.67 (SD= 1.14), a significant improvement in student teachers CRK for teaching systems thinking was 
measured, F (3,103)=4.916, p=. 003, rj 2 =. 125. Looking at the data in more detail (see Table 2), we found the 
improvement was mainly in the knowledge form of student teachers CK for teaching systems thinking. As in the 
pretest, only 8 of 71 student teachers in the intervention courses noted that the definition of a system was 
incorrect in the video and 13 student teachers were able to identify the incorrect definition after the intervention 
(follow-up: 12). The second CK aspect was noted by 17 participants in the pretest, and 26 student teachers were 
able to identify the incorrect labels of the connections and interaction in the posttest. In the follow-up test, the 
number increased a little to 28. Before the intervention, only 5 students noted that the teacher in the videotaped 
lesson showed the complexity of a system mainly as a food web, whereas 17 student teachers identified this 
aspect after the intervention. Remarkably, the number of participants who noted this aspect nearly halved to only 
9 in the follow-up test. 
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Table 2. Identification rate of teacher’s CK errors in the video across three test periods 

Type of CK error 

pretest (tl) 

posttest (t2) 

follow-up test (t3) 

Incorrect definition of a system 

8(11%) 

13 (18%) 

12 (17%) 

Incorret labeling of the connections and interactions 

17 (24%) 

26 (37%) 

28 (39%) 

System complexity is demonstrated mainly as a food web 

5 (7%) 

17 (24%) 

9 (13%) 


Furthermore, the data revealed that the identified mistakes in teachers’ PCK for teaching systems thinking 
remained nearly unchanged across the three test periods. Only the third aspect (planned goals are not achieved) 
revealed an improvement. After the intervention, more student teachers were able to note that the teacher had 
assumed the pupils’ knowledge in systems thinking (18) than in the pretest (3). 


Table 3. Identification rate of teacher’s PCK errors in the video across three test periods 


Types of PCK error 

pretest (tl) 

posttest (t2) 

follow-up test (t3) 

Assumption that the pupils know the term system in ecological 

sciences 

6 (8%) 

3 (4%) 

i (i%) 

Confusing map with many labeled connections and interactions on the 

board 

50 (70%) 

49 (69%) 

51 (72%) 

The planned goals are not achieved 

3 (4%) 

18 (25%) 

18 (25%) 


(2) Are there identifiable differences in the intervention courses? 

Which seminar is most effective in promoting student teachers’ reflective CRK in evaluating a lesson with the 
goal of fostering systems thinking? 

To test statistically which of the four experimental conditions was most effective for increasing student teachers’ 
reflective CRK in evaluating a lesson with the goal of fostering systems thinking, we computed t-tests for 
analyzing the data between different measurements within each group. The analysis of each group are shown in 
Table 4 for the posttest and in Table 5 for the follow-up test. Cohen’s d estimates of effect sizes ranged from 
0.20 to 0.52, suggesting that the treatment had a small effect (Cohen, 1988). 


Table 4. Pretest and postest mean comparisons of student teachers’ CRK 



pretest 

M (SD) 

posttest 

M (SD) 

gain 

M (SD) 

T 

P 

effect 

size d 

Control Group 

0.90 (0.83) 

1.06 (0.97) 

0.16(0.14) 

0.96 

.343 

0.20 

(1) Technical Course 

0.78 (0.85) 

1.26(1.32) 

0.38 (0.34) 

1.42 

.171 

0.44 

(2) Didactic Course 

1.60(1.25) 

2.16(1.52) 

0.56 (0.31) 

1.83 

.080 

0.40 

(3) Mixed Course 

1.36 (0.93) 

1.87 (1.32) 

0.51 (0.23) 

2.23 

.036 

0.48 

ible 5. Pretest and follow-up mean comparisons of student teachers’ CRK 


pretest 

follow-up 

gain 

T 

P 

effect 


M (SD) 

M (SD) 

M (SD) 



size d 

Control Group 

0.90 (0.83) 

0.89 (0.70) 

0.03 (0.11) 

0.24 

.812 

0.00 

(1) Technical Course 

0.78 (0.85) 

1.22 (0.90) 

0.44 (0.27) 

1.64 

.116 

0.49 

(2) Didactic Course 

1.60(1.25) 

1.92 (1.26) 

0.32 (0.28) 

1.14 

.266 

0.24 

(3) Mixed Course 

1.36 (0.93) 

1.88(1.14) 

0.52 (0.25) 

2.08 

.049 

0.52 
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The Mixed Course, which combined technical and subject-related didactic content in almost equal proportions, is 
the only experimental condition that achieved a significant increase of student teachers’ reflective CRK in the 
posttest, t (22)=2.23, p=.036, <7=0.48, and in the follow-up test, t (22)=2.08; p=. 049; d= 0.52. Figure 6 illustrates 
the results of each group. 


2.5 


2 


1.5 


1 
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0 
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■ pretest 
posttest 

■ follow-up Lest 


Figure 6. Results of the treatment groups 


4. Discussion 

“Education is critical for promoting sustainable development and improving the capacity of the people to address 
environment and development issues” (UNCED, 1992). Notwithstanding, knowledge of the dynamics and 
complexity of living systems in biology and geography does not ensure that teachers know how systems thinking 
can be fostered in pupils. 

In an intervention during the summer term 2013, 71 student teachers were trained in different forms of 
knowledge for teaching systems thinking. The intervention contained three varied courses: 1) a course that 
focussed on promoting mainly CK for teaching systems thinking, 2) a course that focussed on promoting mainly 
PCK for teaching systems thinking, and 3) a course that promoted both CK and PCK aspects for teaching system 
thinking in equal parts. 

The use of videos to capture student teachers’ CRK allows not only the measurement of declarative knowledge 
but also the measurement of knowledge in action, e.g., procedural knowledge and reflective knowledge. In the 
pretest analysis, we found just a very low level of student teachers’ CRK in teaching systems thinking. These 
results are in line with the measurement of student teachers’ PCK in the other subsurveys of our larger study 
(Rosenkranzer et ah, 2016). 

The results show an improvement after the intervention by adjusting the pretest score as a covariate. In the 
posttest analysis, the results failed the significance test marginally, but in the follow-up test a significant 
difference emerged, albeit with a low effect size. A similar pattern of results occurered in the category analysis 
of CRK, that is, an increase in student teachers’ reflective CK and PCK for teaching systems thinking. Following 
the results of Oser, Salzmann and Heinzer (2009), declaring video as an effective measurement instrument for 
assessing teachers’ work, we imply an effective intervention in promoting teachers’ reflective CRK for teaching 
systems thinking. The intervention is also consistent with the aims of AGENDA 21 by promoting systems 
thinking in science teacher education. 
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A further goal of our study was to find the most effective intervention course. In teacher education, one 
important ongoing debate is about an adequate CK and PCK base for upcoming teachers. Researchers argue very 
differently over the extent of technical and didactic input (Ball et al., 2005; Borko & Putnam, 1996; Krauss et al., 
2006; Kunter & Baumert, 2013). The findings from our study suggest that the Mixed Course with an equal 
proportion of technical and didactic content will achieve significant improvement. 

Nonetheless, the study has limitations, and caution should be taken to not overgeneralise the findings. Although 
the sample size in each experimental group is relatively small (n<30), the inessential need to randomly assign 
student teachers to experimental courses allows for an improvement in future experimental designs. Another 
limitation is the collection and analysis of the qualitative data. Student teachers’ reflective CRK was obtained 
from their notes, and transforming qualitative dato to quantitative data could have produced erromeous results, 
although our method is based on current statistical methods. 

This study employed digital media to assess teaching sequences in action as a method to improve teacher’s 
knowledge in education. Given the current available options in digital media, traditional research in teacher 
education using pen and paper surveys may no longer be an effective means for measuring the complex use of 
teachers’ professional knowledge in action. Thus, we encourage future research in teacher education to not only 
use pen and paper tests but also digital media to capture teachers CRK and to uncover the complexity of 
teachers’ actions in the classroom. Finally, we hope that the empirical evidence provided by our study will 
inform these discussions. 
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